Vacuity in Testing
نویسندگان
چکیده
In recent years, we see a growing awareness to the importance of assessing the quality of specifications. In the context of model checking, this can be done by analyzing the effect of applying mutations to the specification or the system. If the system satisfies the mutated specification, we know that some elements of the specification do not play a role in its satisfaction, thus the specification is satisfied in some vacuous way. If the mutated system satisfies the specification, we know that some elements of the system are not covered by the specification. Coverage in model checking has been adopted from the area of testing, where coverage information is crucial in measuring the exhaustiveness of test suites. It is now time for model checking to pay back, and let testing enjoy the rich theory and applications of vacuity. We define and study vacuous satisfaction in the context of testing, and demonstrate how vacuity analysis can lead to better specifications and test suites.
منابع مشابه
A Framework for Ranking Vacuity Results
Vacuity detection is a method for finding errors in the modelchecking process when the specification is found to hold in the model. Most vacuity algorithms are based on checking the effect of applying mutations on the specification. It has been recognized that vacuity results differ in their significance. While in many cases vacuity results are valued as highly informative, there are also cases...
متن کاملTemporal Antecedent Failure: Refining Vacuity
We re-examine vacuity in temporal logic model checking. We note two disturbing phenomena in recent results in this area. The first indicates that not all vacuities detected in practical applications are considered a problem by the system verifier. The second shows that vacuity detection for certain logics can be very complex and time consuming. This brings vacuity detection into an undesirable ...
متن کاملA Framework for Inherent Vacuity
Vacuity checking is traditionally performed after model checking has terminated successfully. It ensures that all the elements of the specification have played a role in its satisfaction by the design. Vacuity checking gets as input both design and specification, and is based on an in-depth investigation of the relation between them. Vacuity checking has been proven to be very useful in detecti...
متن کاملVacuity in practice: temporal antecedent failure
Different definitions of vacuity in temporal logic model checking have been suggested along the years. Examining them closely, however, reveals an interesting phenomenon. On the one hand, some of the definitions require highcomplexity vacuity detection algorithms. On the other hand, studies in the literature report that not all vacuities detected in practical applications are considered a probl...
متن کاملHow Vacuous Is Vacuous?
Model-checking gained wide popularity for analyzing software and hardware systems. However, even when the desired property holds, the property or the model may still require fixing. For example, a property φ: “on all paths, a request is followed by an acknowledgment”, may hold because no requests have been generated. Vacuity detection has been proposed to address the above problem. This techniq...
متن کامل